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ABSTRACT 


This research investigated the effects of concurrent 
operator motor loading on performance of a voice recognition 
system comprised of a human operator and a discrete utterance 
voice recognition system. Increased concurrent operator motor 
workload (with respect to that experienced during training of 
the voice recognition system) was found to degrade system 
performance. Operator motor loading was manipulated using a 
rotary pursuit tracker. A special vocabulary was used to 
ensure a baseline recognition error rate to facilitate detec- 
tion of factors influencing system performance. The results 
using the special vocabulary also indicated the performance 
degradations that a real world operator may encounter when 
using different phrases that are similar to one another in 


sound. 
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I. BACKGROUND 


A. INTRODUCTION 

The use of voice to input data to machines has recently 
become technically and practically feasible. The range of 
potentially advantageous commercial and military applications 
of this medium appears extensive. Many applications have al- 
ready been successfully tested and many others are currently 
being investigated; examples of each type are access control 
and command and control by senior officers respectively. 

Commonly the operator of such a system inputs data by 
voice while simultaneously performing a variety of other tasks; 
this feature is one of the chief advantages of voice input. 
The operator is usually not performing these or any other 
tasks when he provides reference samples of his speech to the 
voice recognition device, i.e., when he "trains" the device 
to recognize his speech. However, additional tasking can in- 
duce psychological stress and task-induced stress has been 
shown to produce changes in human speech (Hecker and others, 
1968). Consideration of the above observations raised the 
following question: 

Do these task-induced stresses Significantly degrade 

performance of the voice recognition system as com- 

pared to when the system was trained and operated 


without simultaneous operator tasking? 
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Published research has not adequately investigated the 
potential effect of task-induced stresses on voice recogni- 
tion performance. Harris, North and Owens (1978) observed 
a statistically significant (p <.05) increase in voice recog- 
nition device error rate with increased operator task load 
(generated with a simultaneous compensatory tracking task) ; 
however, their research was not primarily concerned with this 
phenomenon and their conclusions must be viewed as tentative 
Since three of their eight subjects were deleted from the 
recognition error rate analysis due to lost data or because 
they failed to generate recognition errors. Skriver's re- 
search (1979) suggested a similar relationship between recog- 
nition error rate and operator task load on a compensatory 
tracking task but his observed differences in error rate were 
not statistically significant (p >.10) and his research, like 
that of Harris, North and Owens, was not primarily concerned 
with this. 

The research reported here was conducted to further the 
studies referenced above. Specifically, it was designed to 
clearly answer the following questions: 

Does concurrent operator motor loading significantly 

degrade performance of a voice recognition system as 

compared to when the system was trained and operated 


without concurrent operator motor loading? 


del. 








The first chapter provides background information; it 
includes brief descriptions of the status of voice recognition 
technology and its applications, the physiology of speech and 
basic descriptive linguistics, speech changes associated with 
task-induced stress, the principles of operation of the voice 
recognition device used in this research, a Threshold Tech- 
nology Inc. Model T600 discrete utterance voice recognition 
system, and a summary section. Chapter Two describes the 
experiment conducted. Chapter Three is a discussion of the 


results. 


B. VOICE RECOGNITION TECHNOLOGY: STATUS AND APPLICATIONS 
The many advantages of using voice to input data to 
machines include (Lea, 1980a; Lea, 1980b; and Lea and Shoup, 

eo 9) 

1. It frees the operator's hands and eyes and allows 
him more freedom of movement and orientation than 
other input modalities; 

2. Speech is the human's fastest and most natural input 
modality and consequently little or no operator train- 
ing 1S required; 

3. It requires no panel space; 

4. It is compatible with telephone and radio communica- 
tron: alad , 

5. Speech is unaffected by weightlessness and less 


affected by high levels of acceleration and mechanical 
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constraint than are conventional mechanical modes 
of machine input, such as pushing buttons, turning 
knobs, etc. 
Several disadvantages to using voice to input data to machines 
also exist but it appears that they can be overcome (Lea, 
1980a; Lea, 1980b; and Lea and Shoup, 1979). The disadvantages 
include: 
1. Speaker variability, due to sex, dialect, experience, 
etc., is considerable; 
2. Speech communication is not private; 
3. Speech communication is subject to environmental 
noise and distortions; and, 

4. Using voice to enter data can be costly and restricted. 
Work in voice recognition by machine commenced in 1952, 
several years after the introduction of the sound spectrograph. 
Since then, dramatic advances in electronic computer technology 

have permitted corresponding advances in voice recognition 
technology and it has become evident that a whole continuum 

of types of voice recognition devices are feasible. At one 
end of the scale are speaker-dependent devices which recognize 
isolated words (words separated by pauses and spoken by the 
person who "trained" the device). At the other end of the 
Scale are speaker-independent devices which would recognize 
task-independent continuous speech (continuous speech on any 
topic spoken by anyone). Today several speaker-dependent 


devices, which recognize isolated words from vocabularies of 
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up to about 900 words with accuracies in some cases exceed- 
ing 99%, are commercially available. Some military and com- 
mercial applications of these available devices which have 
been investigated or are actually operative are: 
ieee Data entry in cartography (Scott, 1978); 
2. Training air traffic controllers (Grady and Hicklin, 
1976) and uSe in actual air traffic control situa- 
tions (Connolly, 1979); 

3. Command and control by senior officers (Poock, 1980); 

4. Spotting key words in monitored conversations (Beek, 
Neuberg and Hodge, 1977); 

5. Access control (Doddington, 1980); 

6. Quality control and inspection (Interstate Electronics 
Corporation brochure, 1979); and, 

7. Voice actuated devices for the handicapped. 

Other applications and more detail of the history and types 
of voice recognizers are given in Lea (1980a), Lea (1980b) 
and Lea and Shoup (1979). 

Current research is directed at lowering the cost of 
devices like those currently available, making them more 
usable over communication links such as telephone, and making 
accurate and versatile continuous speech recognizers avail- 
able commercially. Lea (1980a) predicts a "rapidly expanding 
market, and a growing variety of applications" for voice 


recognition devices. 


14 











C. PHYSIOLOGY OF SPEECH PRODUCTION AND DESCRIPTIVE LINGUISTICS 

This section was included to define terms used in following 
sections and to emphasize the complexity of speech creation. 
The definitions given were taken from Flanagan (1972) and Lea 
(1980a). 

Figure 1 shows some of the basic physiological elements 
used in the creation of human speech. These elements were 
primarily designed for chewing, swallowing and breathing and 
have evolved the secondary voluntary speech function (Greene 
1964). 

The energy source for speech creation 1s air exhaled from 
the lungs. This air flow may be made to cause the vocal cords 
to vibrate, as follows. Consider the vocal cords initially 
closed so that no path exists for air to exit the lungs. As 
the diaphragm forming the base of the lung cavity is raised 
the air pressure below the cords (subglottal pressure) in- 
creases until finally the cords are forced apart. As the air 
flow through the resulting opening (glottis) increases, the 
local pressure is reduced according to the Bernoulli rela- 
tion, and a force acts to return the cords to the closed or 
proximate position. As the cords are again drawn together 
the air flow is decreased and the local pressure approaches 
the subglottal value. The cycle then repeats itself. This 
results in a cyclic vibration called phonation and a series 
of pulses of air (called glottal pulses) to excite the acous- 


tic system above the vocal cords (supraglottal system). 





SUPRAGLOTTAL SYSTEM 
nasal cavity 
Naar parece 
solf palate (velum) 
oral cavity 
tongue 
pharynx 


VOCAL CORDS 
(in larynx) 


SUBGLOTTAL SYSTEM 


tii eet 


NN 
i 


,» Lungs 
diaphragm 
FIGURE 1. PHYSIOLOGICAL ELEMENTS USED IN 
THE CREATION OF HUMAN SPEECH 






(Adapted from Lea, 198&a) 


16 





Sounds for which the vocal cords are periodically vibrating 
are called voiced sounds; the frequency of vibration of the 
cords is called the fundamental frequency of voicing, Fy 
Sounds for which the vocal cords are not periodically vibrat- 
ing are called unvoiced or voiceless sounds. 

Another means of producing audible sound is to cause the 
airflow to pass through a narrow constriction at some point 
which creates turbulence in the flow; this is called frication. 
The sound produced will depend on the point and manner of 
construction. For example, the tongue can be placed against 
the upper teeth or against the roof of the mouth (hard and 
soft palates) at various locations. 

Resonators, or adjustable cavities such as the nasal 
cavity, oral cavity, pharynx, and trachea, serve to accent 
various frequencies of the sound; peaks in the frequency 
Spectrum of the sound thus produced are called natural reson- 
ances or formants of the speech signal. The first formant, 
Fie usually ranges from 200 to 1000 Hz, the second formant, 
Fo, from 750 to 2700 Hz and the third formant, Fy above 2200 
Hz. Other formants may exist but are usually weak and of 
lesser importance in the analysis of speech signals than the 
first three. 

A language consists of a finite number of distinguish- 
able and mutually exclusive basic sound elements called 
phenemes. Phenemes have the property that if one replaces 


another in an utterance, the meaning of the utterance is 
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changed. The permissable variations of any particular phoneme 
are called allophones; two different phonemes may not have the 
same allophone. 

Each language has its own set of phonemes and features of 
speech which are phonemically distinct in one language may not 
be phonemic in another. English has 46 different phonemes 
(Gleason, 1965) divided into two main groups, vowels and 
consonants. 

The vowel sounds of English are normally produced exclusive- 
ingeoy voiced excitation of the vocal tract. In normal articu- 
lation the tract iS maintained in a relatively stable con- 
figuration during most of the sound; the particular vowel 
uttered depends on the position of the tongue hump, which is 
often the place of greatest constriction, and the degree of 
the constriction. The vowels are also characterized by 
negligible nasal coupling and by radiation only from the 
mouth. 

The consonants are those sounds which are not exclusively 
voiced and mouth-radiated from relatively stable tract con- 
figurations. In general they are characterized by greater 
tract constrictions than the vowels. Continuants are those 
consonants which do not require vocal motion and thus may be 
uttered as sustained sounds as the vowels may be. 

Fricative consonants (or fricatives) are produced from 
noise generated by turbulent air flow at some point of 


constriction of the vocal tract. Common constrictions for 
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producing fricatives are those formed by the tongue behind 

the teeth (dental), the upper teeth on the lower lip (labio- 
dental), the tongue to the gum ridge (alveolar), the tongue 
against the hard or soft palate (palatal or velar, respective- 
ly), and the constricted and fixed vocal cords (glottal). 
Fricatives are continuants and normally radiated from the 
mouth. They can be either voiced (if the vocal cords operate 
in conjunction with the noise source) or unvoiced (if the 
vocal cords do not operate in conjunction with the noise 
source). 

Stop consonants require vocal tract dynamics in their 
creation. A complete closure of the tract is formed (labial, 
alveolar, palatal or velar) and pressure is built up below 
this closure. The stop consonant is uttered when this pres- 
sure is suddenly released and may be either voiced or unvoiced. 

Nasal consonants (or nasals) are continuants and voiced 
and most of the sound is radiated from the nostrils. Just how 
the oral cavity is closed (by the lips or by the tongue at 
the gum ridge or hard or soft palate) influences the oral 
cavity reSonance and hence the sound radiated. 

Glides and semivowels are two groups of voiced consonants 
which greatly resemble vowels; they are characterized by no 
effective nasal coupling and sound radiation from the mouth. 
The glides are dynamic sounds which precede a vowel and 
exhibit movement towards the vowel. The semivowels are 


continuants in which the oral channel 1S more constricted 


ae, 





than in most vowels and the tip of the tongue is not 
down. 

Dipthongs and affricates are combinations of vowels and 
stop-fricatives respectively; both involve vocal tract motion. 
Diagrams showing the location of the major physiological 
elements during creation of each phoneme may be found in most 
texts on speech or linguistics (for example, Flanagan (1972) 
and Gleason (1965)). 

Two speech utterances comprised of exactly the same two 
sequences of allophones may still appear to be very different. 
This may be due to differences in the stress levels on various 
syllables and their distribution throughout the utterances, 
differences in the intonation or systematic variation of pitch 
throughout the utterances, or differences in the rhythm and 
pauses used. These features of speech that go beyond the 
phonemic features are called suprasegmentals or prosodic 
features. 

Speech is not simply the result of many isolated movements. 
It is an extremely complex voluntary function involving the 
well-coordinated and delicate movements of many organs. The 
next section will describe some of the changes in this complex 


function that are associated with task-induced stress. 


D. SPEECH CHANGES ASSOCIATED WITH TASK-INDUCED STRESS 
Psychological stress, however induced, produces a variety 


of changes in human speech; these changes vary considerably 
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from person to person. Many researchers have tried, with 
varying success because of inter-subject variability, to use 
knowledge of these characteristic changes to estimate subject 
stress through analysis of his voice communication. For exam- 
ple, Kuroda and others (1976) discussed a method for determin- 
ing pilot stress and Khachaturyants and Grimak (1972) and 
Kuznetsov and Lapayev (1975) discussed determination of 
cosmonauts' stress. 

One of the most consistent and extensively studied changes 
in speech which accompany increased stress is a rise in the 
fundamental frequency. Analysis of pilots' radio transmissions 
during serious flight difficulties compared with transmissions 
made before such difficulties were encountered has shown this 
relationship between stress and fundamental frequency (Williams 
and Stevens, 1969). Various researchers have attempted to 
use this relationship to determine when subjects were lying 
and when they were telling the truth (for example, Streeter 
and others, 1977); these attempts have usually been only 
partially successful because the relationship did not hold 
for all subjects. Hauser (1975), studying the use of funda- 
mental frequency variations to identify stress arising in 
psychological counselling, found that both higher levels and 
increased variation of fundamental frequency commonly charac- 
terized stressful speech. 

Attenuation of an 8 to 14 Hz microtremor in the speech 


signal is another extensively studied change which is said 
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to be associated with psychological stress (Smith, 1977). A 
device called the Psychological Stress Evaluator (PSE) is 
commonly used to detect and analyze these microtremors. The 
PSE is probably best known for its use in lie-detection 
Situations, although the appropriateness of this has been 
questioned because of several serious methodological problems 
(Brenner and Branscomb, 1979). Schifflet and Loikith (1980) 
recently successfully used the PSE with an automated scoring 
technique to detect stress in the speech of subjects perform- 
ing a four-choice information processing task at different 
rates. Schifflet and Loikith appear optimistic that their 
procedure provides an objective, reliable, sensitive and non- 
obtrusive measure of stress that can be used in the assessment 
of operator workload in vocal communication systems. 

The paper by Hecker and others (1968) is one of the few 
articles specifically addressing speech changes associated 
with task-induced stress. The changes they observed appear 
to this author similar at least in type to the changes ob- 
served in other research where the stress was not task- 
induced. Their subjects were required to read six gauges, 
sum the values, orally announce the sum and read a short 
phrase. Stress was induced by progressively decreasing the 


illumination period of the gauges. Their results indicate 


22 








that task-induced stress produced several characteristic 
changes in the speech signals. Most of these changes were 
attributable to modifications in the amplitude, frequency and 
detailed waveform of the glottal pulses. Other changes re- 
sulted from differences in articulation. Although the manifes- 
tations of stress varied considerably from subject to subject, 
some consistent effects were exhibited by most subjects. Some 
of the observed differences between speech in the control and 
stress condition were: 

1. Differences in volume, in both directions; 

2. Overall differences in fundamental frequency, in both 
directions; 

3. Differences in the time contour of the fundamental 
frequency (for example, in one condition (stress or 
control) the fundamental frequency tended to increase 
from beginning to end of an utterance but in the other 
condition the fundamental frequency tended to decrease 
from beginning to end of the same utterance); 

4. Differences in the precision of articulation (for 
example, some subjects slurred syllables together or 
omitted some speech sounds altogether when stressed); 

5. Differences in the amount of high frequency energy in 
the glottal pulses and in voiceless consonants; 

6. Differences in monotonicity of the speech (some of 
the subjects tended to speak in more of a monotone 


when they were stressed); and, 
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7. Differences in the regularity of glottal vibration 
(stressed speech tended to have more irregularity 
Pie eeerateernmor Glottal Vibration, referred to as 
vyolcing leregqularity) . 

Most of the speech changes described in this section were 
detected by sophisticated electronic equipment. However, at 
least some changes were apparent to human listeners as several 
of the studies referenced found that independent human listen- 
ers could often correctly identify, with better than chance 
accuracies, which speech was stressed and which was not. This 
observation is consistent with the author's, and probably most 
readers', personal experience; the author can recall numerous 
occasions on which the speech of some temporarily frustrated, 
excited or frightened acquaintance distinctly changed in some 
manner. 

E. PRINCIPLES OF OPERATION OF THE THRESHOLD TECHNOLOGY INC. 
MODEL T600 DISCRETE UTTERANCE VOICE RECOGNITION SYSTEM 
The speech recognition system used in this research was 

a Threshold Technology Inc. Model T600 discrete utterance 

voice recognition system, a speaker-dependent isolated utter- 

ance recognizer. The T600 could be trained to recognize 

any continuously spoken utterance (i.e., not including pauses) 

of between .1 second and two seconds duration. An utterance 

could thus be a single word, a continuous string of several 


words, or a continuous string of nonsense sounds not necessarily 
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comprising any known word or words. The T600 used ieee S 
research had several added memory modules which allowed use 
of vocabularies of up to 256 utterances. Utterances spoken 
for recognition by the T600 had to be separated by pauses of 
at least .1 second duration or else the T600 "connected" the 
utterances and tried to recognize the "connected" utterances 
as one utterance. 

An utterance spoken into the special T600 noise-cancelling 
microphone was processed as follows. The signal from the 
microphone was passed through 19 bandpass filters spanning 
the speech spectrum; the filters were spaced non-linearly to 
minimize the number required. A spectral shape detector then 
calculated rate of change of energy level with respect to 
frequency information to describe the overall signal spectral 
shape. The spectral shape and its changes over time were 
continuously calculated and these measurements were processed 
to determine presence or absence of each of 32 acoustic 
features; this binary feature information was produced every 
two milliseconds. The features were of two main types. About 
half of them were related to the relative energy content of 
particular spectral bands. The other half resulted from 
logical and analog operations on the short-term power spect- 
rum and most were attempts to automatically detect phonemic 
information. 

When the end of the utterance was detected, by means of 


the pause which separated consecutive utterances, the duration 


25 








of the utterance was divided into 16 time segments and the 
features were reconstructed into a normalized time base. This 
process resulted in a 512 bit - 32 binary features by 16 time 
segments - description of the utterance. 

During the training of the T600 the subject repeated each 
utterance of the vocabulary ten times; ideally the ten repeti- 
tions of an utterance were representative of the variety of 
ways the subject normally said the utterance. The T600 extract- 
ed a 512 bit feature matrix for each version of each utter- 
ance. The ten matrices for each utterance were then combined 
to produce a single reference matrix for each utterance; an 
element ais of the reference matrix for an utterance indicated 
presence of feature i at time segment j only if feature i was 
present at time segment j in a minimum number of the original 
ten matrices for that utterance. (Threshold Technology has 
not disclosed the value of this minimum number.) 

When an utterance was spoken for recognition by the T600 
a 512 bit descriptive matrix was calculated and weighted cor- 
relations between this matrix and each of the reference 
matrices describing the vocabulary utterances were calculated. 
The vocabulary utterance which resulted in the largest cor- 
relation exceeding some preset threshold value was then 
selected as the utterance which the T600 "thought" was spoken. 
If no correlation exceeded this preset threshold value the 
T600 emitted a "beep" sound which signalled the operator to 


try again. 
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The T600 functioned similarly to the older Threshold 
Technology Inc. VIP 100 which was described in some detail by 
Porter and others (1977) and Martin and Grunza (1974). 
Recognition accuracies obtained with the VIP 100 have often 
been better than 99% as, for example, in the studies by Martin 
and Grunza (1974), Scott (1975) and Scott (1978). 

Several other discrete utterance recognizers operate in 
a Similar manner to the T6000; often different descriptive 
features of the speech are extracted and/or a different num- 
ber of time segments used. Other recognizers operate dif- 
ferently. Lea (1980a) and Lea (1980b) provide summaries of 
available detail on commercially available recognizers and 
on other recognizers still in the developmental or prototype 


stages. 


F. SUMMARY 

The previous pages have briefly described the physiology 
of speech, various features which characterize the acoustic 
speech signal, changes in these features associated with task- 
induced stress, and how the successful operation of a partic- 
ular speech recognition device depends on extraction of some 
of these descriptive features from the speech signal over time 
and comparison of the resulting representation of the speech 
with previously stored reference representations of the operat- 
ing vocabulary. The author hypothesizes that if operator 


tasking is significantly different during training and operation 


on 
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of the recognizer, as if often the case, then the operator's 
speech will not be the same during training as during opera- 
tion and consequently performance of the recognition system 
will be less than optimal. The fact that the changes in speech 
associated with task-induced stress may be different in nature 
and/or direction for different operators, which has limited 
the success of determining subject stress through analysis of 
his speech, does not obviate this argument. Certainly dif- 
ferent changes and different directions of change may cause 
different degrees of decrement and the degree of decrement 

may also be different for different recognition devices be- 
cause of differences in the descriptive features extracted 
and/or algorithms used; nevertheless, the decrement should 
occur to some degree with all recognition devices and with all 
operators whose speech changes in some manner. 

The objective of this research was to begin to investigate 
this expected decrement in recognition system performance. In 
particular, this research attempted to clearly answer the 
following question, which was posed at the beginning of this 
chapter: 

Does concurrent operator motor loading significantly 

degrade performance of a voice recognition system as 

compared to when the system was trained and operated 


without concurrent operator motor loading? 


28 





It is emphasized that this research was an investigation 
into a phenomenon which should be common to all voice recog- 


nition systems. 


Zo 





II. DESCRIPTION OF THE EXPERIMENT 


ie ODJIECTIVE 

The objective of this experiment was to determine if con- 
current operator motor workload affected the performance of a 
voice recognition system comprised of a human operator and a 
discrete utterance voice recognition device. A special vocab- 
ulary was used to ensure a baseline error rate with which to 
compare various motor loading levels. As such, it was expected 
that absolute error rates would be higher than those normally 


realized in real world operations. 


Be OUBJECTS 

Twenty-four subjects participated on a volunteer basis 
with no monetary or other incentive. Twenty of the subjects 
were male military students at the Naval Postgraduate School 
(NPS), three were male, military staff members at NPS, and 
one was a female civilian employee of NPS. The military 
officers represented the United States Navy, Army, Marine 
Corps and Coast Guard and their ranks ranged from Lieutenant 
to Lieutenant-Commander and from Captain to Major inclusive. 
All subjects were between the ages of 26 and 38 incluSive. 

None of the subjects had any previous experience on the 
voice recognition system used in the experiment. Only three 
of the subjects - numbers 1, 6 and 13 - had any experience - 


two hours each - on pursuit tracking devices, the type of 
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device used to create operator motor loading. 


C. EQUIPMENT USED 
1. Pursuit Tracker 

A Lafayette Instrument Co. Model 2203E Photoelectric 
Pursuit Tracker was used to induce operator motor loading. 

The tracker was positioned on a table in the booth so that the 
pursuit surface was 34.25 inches above the floor. Subjects 
stood while performing the tracking task and were required to 
track an approximately .75 inch square light target with a 
13.06 inch long wand. The target travelled clockwise at a 
constant 40 rpm around both of two paths used. One path was 
circular with outer diameter 12 inches; the other, detailed 

in figure 2, was square-like. Figure 3 is a photograph of the 
tracker in the booth (described later). A camera exposure 
time of 1.5 seconds was used for the photograph so that the 
target would travel completely around and illuminate the whole 
Path. 

Time on target (TOT), the performance score for the 
tracking tasks, was recorded using a Lafayette Instrument Co. 
CraLab Model 172 timer and task duration was controlled using 
a Lafayette Instrument Co. Model 54014 Stop Clock. Both of 
these devices were located outside of the booth at the experi- 
menter control station. 

Harris (1978) reviewed research reported from 1967 to 


1977 inclusive on concurrent verbal and tracking tasks. He 
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FIGURE 2. SQUARE-LIKE TRACKER PATH 
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concluded that "it is clear that performance of some verbal 
tasks interferes with simultaneous performance of some track- 
ing (control) tasks" and that "it is probable that certain 
parameters of the control task will be important determinants 
of any decrement in performance observed during simultaneous 
verbal information processing." Subsequent research by Harris, 
North and Owens (1978) involving inputting data with a voice 
recognition device and compensatory tracking found decrements 
in performance of both of these tasks when performed simultan- 
eously versus when they were performed separately. For example, 
they found that when the voice input and tracking tasks were 
performed concurrently recognition error rate was approximately 
55% greater and tracking RMS error approximately 30% greater 
than when these tasks were performed separately. 
2. Voice Recognition System and Choice of Vocabulary 

A Threshold Technology Inc. Model T600 discrete utter- 
ance voice recognition system (which will hereafter be refer- 
red to as the T600) was used as the equipment component of 
the combined equipment plus human operator voice recognition 
system. A description of the operation and capabilities of 
this equipment was provided in Chapter One. 

The vocabulary used in this experiment consisted of 
50 different utterances. Thirty were single words selected 
by the experimenter from the Listener's Answer Sheets of the 
Modified Rhyme Test, one of the four test types which have 


been commonly used in measuring intelligibility in speech 
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communication (Kryter, 1972). Sixteen of these 30 words 

were eight pairs of rhyming words which, within each pair, 
differed only with respect to initial consonant - for example, 
"beat" and "peat". The other 14 words were seven pairs of 
non-rhyming but similar words which, within each pair, dif- 
fered only with respect to final consonant - for example, "Sap" 
and "sat". The other 20 utterances were chosen by the experi- 
menter from single words commonly used in Command and Control 
environments; they were chosen to be more easily distinguished 
from each other and from the other 30 words of the vocabulary. 

All words of the vocabulary were one or two syllables 
in length. Short words were deliberately selected to facili- 
tate generation of as many T600 word recognition attempts as 
possible in the limited time that each volunteer subject was 
available. The vocabulary is listed by word type in Appendix 
A. A listing in the order in which the words were trained is 
attached to the written’ instructions initially given to sub- 
jects and is contained in Appendix C. 

This particular vocabulary was chosen to increase the 
likelihood of recognition errors by the T600 for the following 
reasons. (T600 recognition errors (RE's) are operationally 
defined in the Dependent Variables section of this chapter.) 
Recognition accuracy with older Threshold Technology Inc. 
voice recognition equipment similar to the T600 and uSing more 
normal vocabularies (i.e., comprised entirely of more easily 


distinguished words) has often been better than 99%, as for 
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example, in the studies by Martin and Grunza (1974), Scott 
epg7>) and Scott (1978). This level of accuracy would pro- 
duce an average of about one (or less) RE's per 100 spoken 
utterances. It was anticipated that if operator motor load- 
ing did affect recognition accuracy then the effect would be 
relatively small and, due to the discrete nature of RE's, 
would probably not be easily distinguishable if only one RE 
per 100 utterances were being observed - for example, a 20% 
increase in RE's would probably not be great enough to produce 
a sufficient number of increased RE observations to be stat- 
istically distinguishable from inherent random variation. 
However, if a vocabulary could be chosen to produce approxi- 
mMately ten RE's per 100 utterances a 20% increase in RE's 
should be more easily distinguishable as this would result in 
an average observation of 12 RE's per hundred utterances. 

An alternative method of detecting a small expected 
change in recognition accuracy would be to increase the num- 
ber of utterances spoken by the subjects. This was not con- 
Sidered feasible here because of the greatly increased time 
which would be required of each of the volunteer subjects; 
the experimental design used required between 1.5 and two 
hours per subject. For this reason, the former method, special 
vocabulary, was used. 

3. Arrangement of Equipment Used 
Figure 4 illustrates the functional relationships 


among the various experimental devices used in the experiment. 
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The subjects were seated (except while tracking) one at a time 
Maman industrial Acoustics Co. Inc. Controlled Acoustic Environ- 
ments booth. The tracker was on a table in front of the 
subject. 

A Maico Model MA-24B Dual Channel Research and Diag- 
nostic Audiometer and headsets were used to provide oral com- 
munication between the subject and the experimenter. The 
experimenter could speak to the subject by depressing a 
"talk-over" switch. Another microphone, placed in the booth, 
was live at all times and permitted the experimenter to hear 
what was happening in the booth - in particular, what the 
subject said. A Sony model TC 124 cassette tape recorder was 
connected to permit simultaneous recording of the signals 
detected by the booth microphone and the signals that the 
subject received over his headset. 

The special T600 system noise-cancelling microphone 
wasS mounted on the subject's headset and connected only to 
the T600. The microphone ON/OFF switch was located outside 
of the booth. 

A Computer Devices Inc. Model 1203 Miniterm portable 
terminal was connected to the T600 system in such a manner 
that when the T600 recognized an utterance the output string 
for that utterance was typed at the terminal. The T600 was 
programmed so that the ASCII output stream associated with 
each utterance of the vocabulary was simply the letters spel- 


ling the utterance followed by a carriage return and a line 
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feed; thus, for example, if in the recognition mode the T600 
"thought" that a subject said "attack", the word "attack" was 
displayed on the CRT on a separate line and printed at the 
terminal, also on a separate line. This provided the experi- 
menter with a paper printout of T600 recognition activity 
which, with the correct utterances recorded on the cassette 
tape recorder, permitted thorough analysis of the data. Ac- 
curate, manual, real-time analysis by the experimenter uSing 
only the T600 CRT was infeasible primarily because of the rate 
at which the T600 was required to process signals for recog- 
nition - one word every three seconds. 

An Akal model 4000DS Mk II reel-to-reel tape recorder 
was connected to the Maico Audiometer and used to present 


stimuli to the subject. 


D. EXPERIMENTAL PROCEDURE 

Subjects were tested one at a time during normal working 
hours. They were first required to complete the Subject Data 
Form (Appendix B) and then read two pages of written instruc- 
tions (Appendix C) which briefly introduced the experiment and 
provided general guidelines on inputting voice data to the 
T600. Remaining instructions to the subject were given orally 
by the experimenter who closely followed the aide-memoire 
Shown in Appendix D. 

Subjects were next given a brief demonstration of the 


Operation of the T600. For this stage the T600 microphone 
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and the headset on which it was mounted were removed from 

the booth and the microphone was reconnected outside of the 
booth so that the subject could immediately see what happened 
when speech signals were input to the T600. The importance 
of the guidelines which the subject had just read were demon- 
strated during this stage and the subject was allowed to fam- 
iliarize himself with the T600 for about five minutes. 

The T600 microphone and the headset on which it was mounted 
were then reconnected inside the booth. The 50 word vocabu- 
lary was then trained one word at a time. The experimenter 
had all of the T600 controls outside of the booth and closely 
controlled the training process, requiring the subject to 
retrain words as necessary - for example, if a word was 
initially trained monotonously. The T600 was next put in the 
recognition mode and recognition of each word of the vocab- 
ulary was checked. The subjects repeated each word of the 
vocabulary three times. Those words which were not recognized 
correctly at least two of the three times were immediately 
retrained, then rechecked and retrained as necessary. If 
either word of a rhyme or non-rhyme but similar word type pair 
required retraining both words of the pair were subsequently 
rechecked for recognition and retrained and rechecked as 
necessary before continuing. 

The subject next received, via his headset, a 2.5 minute 


tape recording of the 50 words of the vocabulary arranged in 
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random order and presented at a constant rate of one word 
every three seconds. The subject was instructed to repeat 
the words one at a time for recognition by the T600. He was 
advised to try to repeat each word and to guess with a word 
in the vocabulary if he was uncertain. 

Next the subject was briefed on the two tracking tasks 
that he would be performing - circular tracking and square- 
like tracking. He was advised that his tracking scores would 
be time on target. He was then given a 2.5 minute practice 
on the circular tracking task. 

The subject was next given a combined 2.5 minute circular 
tracking and word repetition task for recognition practice. 
The subject was played the same 2.5 minute tape recording 
that he had heard earlier and was instructed as before to 
repeat the words one at a time for recognition by the T600. 

He was advised that word recognition was the higher priority 
task but that he was to simultaneously perform the tracking 
task as well as he could with whatever capabilities he had 
remaining after attending to the priority task. The subject 
was also reminded to be sure to repeat each of the taped words 
and to guess with a word in the vocabulary if he was uncertain. 

The subject was then exposed to the three experimental 
conditions corresponding to the three operator motor loading 
conditions - no tracking task (NTT), circular tracking task 
(CTT), and square-like tracking task (STT). These were de- 


Signed to create different levels of operator motor loading. 
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Each condition lasted five minutes and each of the 24 subjects 
received the three conditions in a different order (see Ap- 
pendix E). 

During condition NTT the subject was required only to re- 
peat two different consecutive random orderings of the words 
of the vocabulary; these were presented to him over his head- 
set as during practice. The first time through the vocabulary 
in any condition was referred to as the first half of the 
trial; the second time was referred to as the second half of 
the trial. he first word of the second half followed the last 
word of the first half with the same spacing used within the 
two halves; the subject received no cues that he was halfway 
through the trial. In each of the conditions CTT and STT 
the subject was similarly required to repeat random orderings 
of the vocabulary (two different orderings for each condition 
as in condition NTT - all random orderings used are shown in 
Appendix F); however, he was also required to perform simul- 
taneously the appropriate tracking task. He was reminded that 
the reception of words for recognition by the T600 was the 
higher priority task and to guess with a word from the vocab- 
ulary if he was uncertain, as during the combined practice. 
(The purpose of this instruction was to ensure that the T600 
received the same, or at least nearly the same, utterances for 
recognition during each trial half and thus provide a common 
basis for comparison of T600 recognition errors.) By monitor- 


ing the T600 CRT display and time on target timer, listening 
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to booth activity via the booth microphone, and post-experi- 
Meme questioning of subjects, the experimenter ensured that 
subjects adhered to the instructions that they had been given. 

Immediately after a subject completed each condition, and 
before he was allowed to leave the booth, he was inStructed to 
complete the "Feeling Tone Checklist" shown in Appendix G in 
accordance with the instructions also shown in Appendix G. 
This checklist, developed by Pearson and Byars (1956), was ad- 
ministered to assess possible differential subjective fatigue 
after each of the three different motor loading conditions. 

During the experimental conditions subjects were not given 
feedback on their tracking performance. During the practice 
sessions the only feedback given to subjects regarding T600 
recognition of their speech was the knowledge of which words 
required retraining; no feedback regarding T600 recognition 
performance was given to subjects during the experimental 
conditions. Those subjects who indicated interest on their 
"Subject Data Sheets" were individually briefed on their track- 
ing performance, T600 recognition of their speech and the 
hypotheses being tested immediately after they completed the 
last experimental condition. 

Subjects were allowed to take short rest breaks as they 
wished during the training and practice sessions and before 
each of the three experimental conditions. A drinking foun- 
tain was located nearby for any subjects who became thirsty 


or whose throats became dry. 
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E. DEPENDENT VARIABLES 

The following were calculated for each half of each trial: 

1. T600 recognition errors (RE's) 

2. Subject verbal errors. 
In this experiment a T600 recognition error was operationally 
defined to be a failure of the T600 to recognize correctly any 
vocabulary word which a subject said; this included both in- 
correct recognition (for example, the subject said "beat" and 
the T600 "thought" he said "peat") and rejection (for example, 
the subject said "dip" and the T600 failed to recognize sie 
and emitted a "beep" sound). This definition is different 
from most definitions of recognition error in the voice recog- 
nition literature which do not include rejections - for example, 
Martin and Grunza (1974). The operational definition used in 
this experiment was considered more consistent with the aim 
of this research - i.e., to answer the question: Would in- 
creased concurrent operator motor workload (with respect to 
that experienced during training of the recognition device) 
result in changes in his speech which would in turn result in 
degraded performance of the voice recognition system? It was 
believed that if the T600 rejected "dip" when said by a sub- 
ject under condition STT, but not when said by the same sub- 
ject under condition NTT, this suggested changes in system 
performance as a result of changes in the subject's speech 


and accordingly should be recorded and analyzed. 


44 





A subject verbal error was defined as a failure of the 
subject to repeat correctly the presented word. This failure 
could be either a failure to respond (omission) or responding 
with a non-vocabulary word or the wrong vocabulary word 
(commission). 

Tracker time on target scores were recorded at the end of 


conditions CTT and STT. 


fee HYPOTHESES 
The following hypotheses were to be tested. 


1. Hypotheses Regarding T600 Performance 


a. HO: The different levels of operator motor load- 
ing would not have different effects on T600 
recognition error rate. 

ees H false. 
It was expected that increased operator load- 
ing would result in increased recognition 
error rate (RER), i.e., RER(NTT) < RER(CTT) 
< JRER (Sia) 

b. H_: The two trial halves would not have different 

effects on T600 recognition error rate. 
Hj: H false. 
2. Hypotheses Regarding Subject Performance 
a. Ho? The different levels of operator motor load- 


ing would not have different effects on sub- 


ject verbal error rate. 
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H 


Ay false. 

It was expected that increased operator load- 
ing would result in increased subject verbal 
Server rates ven), 1.e., VER(NTT) < VER(CTT) 

<)> VERUSTT) 

(This hypothesis was suggested by the research 
of Johnston (1975) who observed a significant 
detrimental effect of a simultaneous compen- 
satory tracking task on speech intelligibility 
in noise.) 

The two trial halves would not have different 
effects on subject verbal error rate. 

Hy false. 

Time on target for the Circular Tracking Task 
would equal time on target for the Square 
Tracking Task. 

Hy false. 

It was expected that time on target for the 
Square Tracking Task would be less because 

in this task the constant 40 rpm angular 
velocity caused the target to move at a con- 
Stantly changing speed whereas in the Cir- 
cular Tracking Task the constant angular vel- 


ocity resulted in a constant target speed. 
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d. H_: Subject subjective fatigue (as measured by 


O 
the "Feeling Tone Checklist" of Pearson and 
Byars, 1956) would be the same for the four 
operator motor loading conditions. 

Hy: HA false. 


It was expected that increased operator load- 
ing would result in increased subjective 
EAGLOUC (St) , 1. eon chanel) < SF (CTT) 
—esksiT) 

Tracker performance was not recorded at the end of the 
first half of trials and hypotheses regarding tracker perform- 
ance versus trial half were not devised. (A test of such 
hypotheses would probably have shown tracker performance to 
be lower in the second half of trials than in the first half, 


due to subject fatigue.) 


G. EXPERIMENTAL DESIGN 

A conceptual design for the experiment is shown in figure 
5. This is a three factor factorial design. 

The three experimental conditions could be presented in 
Six (three factorial) possible orders. Each order was used 
with four subjects. Subject to this restriction the order of 
presentation of the three conditions to any particular sub- 
ject was assigned randomly. 

Subject verbal error rate and T600 recognition error rate 
data were expected to be inherently Binomial in nature. In 


the case of subject verbal errors, the values of p, the 
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probabilities of a subject verbal error, or equivalently, 
subject verbal error rates, were expected to be small. Be- 
cause of this and because the values of n, number of words to 
be spoken, were relatively large, it was concluded that the 
distributions of subject verbal errors could be approximated 
by Poisson distributions and statistical methods based on the 
Poisson distribution were selected to test subject verbal 
error rate hypotheses. 

In the case of T600 recognition error rates, the values of 
p, probabilities of a recognition error or recognition error 
rates, were expected to be too large to permit analyses based 
on the Poisson distribution. It was decided that a parametric 
analysis of variance would be used to test recognition error 
rate hypotheses; prior to this analysis the data would be 
transformed using the arcsin transformation, y' = 2arcsin 
ae), to remove the relationship between the variance and 
mean expected because of the Binomial nature of the data. 

Non-parametric tests were selected for testing hypotheses 
regarding tracking scores and subjective fatigue because 
these data were not expected to meet the assumptions of para- 
metric tests. 

Because of the exploratory nature of this research, a level 
Seesuoniticamce, «a, of .10 was Selected during the design 


phase. This value was used in all tests of hypotheses. 
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Hee RESULTS 
1. Results for T600 Performance 

Appendices H, I, J and K present separate confusion 
matrices for each of the three operator motor loading - 
experimental conditions (NTT, CTT and STT) and for all three 
conditions combined respectively. A matrix element ais of 
these matrices indicates the proportion of the time that the 
T600 "thought" that a subject said word j when the subject 
actually said word i. Appendix L shows total T600 recognition 
errors for each subject for each half of each trial under each 
operator motor loading condition. Mean T600 recognition error 
rates for each operator motor loading condition, trial half 
and vocabulary word type, expressed in recognition errors per 
100 spoken utterances, are shown in table l. 

Figure 6 is a plot of the recognition error rate 
observations and figure 7 a plot of the arcsin transformed 
recognition error rate observations. Figure 7 shows that the 
parametric analysis of variance homogeneity of variance assump- 
tion was adequately met. Since the parametric analysis of 
variance is quite robust regarding its Normality assumption 
(Scheffé, 1959), it was felt that this assumption also was 
adequately met and a parametric analysis of variance was per- 
formed on the arcsin transformed data. The results are sum- 


marized in table 2. The model for this analysis was: 
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TABLE 1 


MEAN T600 RECOGNITION ERROR RATES* 


BY OPERATOR MOTOR LOADING - EXPERIMENTAL CONDITION 


(over all word types and both trial halves) 


NTT 0. > iss 
CLT 14.43% 
Sit 14.73% 


BY TRIAL HALF (over all word types and motor loading conditions) 
Birst halt L2e713% 


Second half 1344933 


BY VOCABULARY WORD TYPE (over all motor loading conditions and 


both trial halves) 


Rhyming 25.673 

Non-rhyming but similar 2 5 os 

Operational 3.48% 
OVERALL P32 28 


* Expressed in recognition errors per 100 spoken utterances. 
A recognition error was operationally defined in this re- 
search to be a failure of the T600 to recognize correctly 
any vocabulary word which S spoke and includes both in- 
correct recognition and rejection of vocabulary words; 
recognition errors do not include those cases where S 
spoke a word not in the vocabulary (or coughed, sighed, 
etc.) and the T600 generated a recognition. 
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TABLE 2 


ANALYSIS OF VARIANCE FOR T600 RECOGNITION ERROR RATE 


Source df 
Subjects 23 
L (Operator motor 2 


loading condition) 


H (Trial half) 1 
ines 2 
Error vs 


* py <.0005 


NS - Not significant, l1.e. p >.10 
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nea < 
where er = arcSin transformed recognition error 
rate for operator motor loading con- 
dition 1, trial half j, and subject k; 
the range of Yaak 1S Ome @ ell", 


u = common experimental contribution to aE 
ie = contribution of operator motor loading 
Condtelon a,  1=152,3 (NGL, CTT, STT) 


Be = COntErtaatronger trialwhalf sj.,j=1,2 
(first half, second half) 

Si =JGontcriouti~onwoL subject k, K=1,2) «su, 24 

Clik = random error 


Subject effects were considered to be random; all others were 
considered to be fixed. 

The analysis showed motor loading to be significant 
(F=13.95, df=2/115, p < .0005). A parametric Range Test (Hicks, 
1973) was performed to determine which operator motor loading 
conditions were statistically different (with respect to T600 
recognition error rates) and it was concluded that the only 
Significant differences (a = .10) were between condition NTT 
and both of the other two conditions, CTT and STT. 

Trial half was not Significant (F=1.65, df=1/115, 

p >.10). The interaction between operator motor loading and 
trial half was not significant (F < 1). 
A non-parametric Friedman two-way analysis of variance 


(Siegel, 1956) was performed and concluded that recognition 
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error rate differed by vocabulary word type (rhyming, non- 
rhyming but similar, and operational) (x “=46.08, adf=2, 

p < .0005). A non-parametric test proposed by Nemenyi (in 
Kirk, 1968, p. 497) was performed to determine which pairwise 
comparisons of recognition error rate were significant; it 
was concluded that all pairwise differences were significant 
(p < .01), i.e., each of the vocabulary word types were dif- 
ferent from each other. 

Figure 8 shows recognition error rate versus operator 
motor loading condition for each trial half for each vocab- 
ulary word type. Figure 9 is a simplified version of figure 
8 showing recognition error rate versus operator motor load- 
ing for each trial half. 

Subjects were instructed to repeat each vocabulary 
word heard and to guess with a word in the vocabulary if un- 
certain of the word. The purpose of this instruction was to 
ensure that the T600 received the same, or at least nearly the 
Same, utterances for recogntion during each trial half, l.e., 
each vocabulary word once, and thus provide a common basis 
for comparison of T600 recognition errors. Despite the in- 
struction instances arose where subjects either did not speak 
any word or spoke a word not in the vocabulary; these are 
tabulated in Appendix M. T600 recognition errors, as opera- 
tionally defined in this research, could not occur in these 
instances and the following adjustment was made to establish 


a reasonably common basis for comparison. If x T600 recogni- 
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FIGURE 8. MEAN T600 RECOGNITION ERROR RATES 


(in recognition errors per 100 spoken utterances) 


57 





MEAN T600 RECOGNITION ERROR RATE (%) 


rs 


14 


13 


2 


11 


re 


/ ate Second half 
Of. trials 


Overall 


Pirst halt 
Ofeste eels 





ee eee) ee 


Ni Car. STT 
OPERATOR MOTOR LOADING CONDITION 


FIGURE 9. MEAN T600 RECOGNITION ERROR RATES 
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tion errors occurred in a particular trial half for a subject 
and that subject made y errors of the above type in the trial 
half, then the error rate observation on which the analysis 
was based was x/(50-y), not x/50. 

2. Results for Subject Performance 

Appendix M shows total subject verbal errors for each 
subject for each half of each trial under each operator motor 
loading condition. Mean subject verbal error rates for each 
motor loading condition, trial half and vocabulary word type, 
expressed in subject verbal errors per 100 words presented to 
the subject for repetition (i.e., each word of the 50 word 
vocabulary twice), are shown in table 3. 

A test based on the Poisson distribution (Cox and 
Lewis, 1966, p. 231-234) was performed on the subject verbal 
error rate data and showed the operator motor loading condi- 
eromeertect to be significant (p < .05, a =.10). 

The hypothesis regarding subject verbal error rate 
versus trial half was not tested as the mean error rate was 
the same for each trial half (see table 3). 

Subject tracker time on target scores are shown in 
Appendix N. A non-parametric Wilcoxon matched-pairs signed- 
ranks test (Siegel, 1956) was performed to determine if track- 
er performance (aS indicated by time on target) was statist- 
ically different when tracking the two different paths. The 
test concluded that time on target scores were statistically 


greater when tracking the circular path than when tracking the 
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TABLE 3 


MEAN SUBJECT VERBAL ERROR RATES* 


BY OPERATOR MOTOR LOADING ~- EXPERIMENTAL CONDITION 


Ne ~29% 
CTT ays 
oii ~54% 
BY TRIAL HALF 
First half 33% 
Second half Soo 
BY VOCABULARY WORD TYPE 
Rhyming - 13% 
Non-rhyming but similar .60% 
Operational ao 
OVERALL wot 


x Expressed in subject verbal errors per 100 vocabulary 
words presented to S via the headset. A subject ver- 
bal error was defined in this research to be a failure 
of the subject to repeat correctly the presented vocab- 
ulary word. This failure could be either a failure to 
respond (omission) or responding with a non-vocabulary 
word or the wrong vocabulary word (commission). 
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square-like path (p <.00006). 

The results of the subjective fatigue enquiry are 
shown in Appendix O. Numerical scores shown were obtained by 
multiplying the number of items scored "better than" by two 
and adding the number of items scored "same as", as recom- 
mended by Pearson and Byars (1956). A non-parametric Friedman 
two-way analysis of variance was performed on this data and 
it was concluded that subjective fatigue was the same for the 
Pimeee operator motor loading conditions (y ; = 1.90, df=2, 

Eee >, a=.10). 
3. Other Results 

The following were investigated graphically: 

a. T600 recognition error rate versus subject 
verbal error rate; and, 

b. Tracker scores versus subject verbal error rates. 
No relationships were apparent. Spearman rank correlation 
coefficients between subject tracker scores and T600 recogni- 
tion error rates were calculated for each tracker condition. 
The coefficient for condition CTT was not significant 
(rx. (CTT) = -.304, r (critical) = +.343, two-tailed test, 
a=.10). However, the coefficient for condition STT was sig- 
nificant (rx. (STT) = -.,401, r(critical) =+.343, two-tailed 
test, a=.10); i.e., there was a Significant tendency for 
subjects to do either well or poorly on both the square track- 
ing and voice input tasks but not poorly on one and well on 


the other. 
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Mite DEoCUS > LON 


Operator motor loading was found to affect significantly 
the performance of the voice recognition system in that mean 
T600 recognition error rates in the two conditions involving 
concurrent tracking were 39% greater than the mean error rate 
of the no tracking condition. Although mean error rates 
appeared greater in the second half of trials than in the 
first half (see table 1 and figure 9), the difference was 
Meesignificant (p > .10). 

The hypotheses regarding T600 recognition error rates 
were retested using operational words only and tests based 
on the Poisson distribution. The trial half effect was found 
to be not significant (p > .5, two-tailed test, a=.10), as 
it was when using the whole vocabulary. The operator motor 
loading condition effect was significant (p <.10, a=.10), as 
it was when using the whole vocabulary. This last result sup- 
ports the following generalization: recognition error rate 
will be greater with concurrent motor loading (like conditions 
CTT or STT) than without (condition NTT) even if the vocab- 
ulary is comprised entirely of operational-like utterances, 
i.e., a real world vocabulary versus the artificial vocab- 
ulary of the experiments (assuming the operator trained the 
system without concurrent motor tasking, i.e., in the usual 


manner). 
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Operator motor loading condition had a Significant effect 
on subject verbal error rate, as expected. Subject verbal 
error rate did not differ by trial half. 

The subjective fatigue checklist failed to disclose sig- 
nificant differences between any of the three operator motor 
loading - experimental conditions. This was probably partly 
because the effects of order of presentation of the condi- 
tions dominated any possible condition effects during subjects 
scoring of the checklists. (Several subjects advised the 
experimenter after a tracking condition that the condition 
was more fatiguing than condition NTT but they had to score 
the tracking condition higher because it was the last, or next 
to last, condition and the subject felt good because the end 
was at hand.) 

It is emphasized that the recognition error rates obtained 
with the T600 in this experiment are at least ten times what 
has commonly been found. These higher recognition error 


rates were deliberately sought by the experimenter to facili- 


tate detection of factors influencing system performance and 
are primarily due to the special vocabulary employed. The 
results uSing the special vocabulary also indicated the 
recognition performance degradations that a real world opera- 
tor may encounter when uSing different phrases that are 
Similar to one another in sound, such as "above glide slope" 


and "below glide slope" or "VHF" and "UHF". 
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The author is continuing to investigate the use of the 
special vocabulary and the effects of task-induced stress on 


performance of a voice recognition system. 
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APPENDIX A 


VOCABULARY LISTING (BY WORD TYPE) 


RHYMING 

gale page gold cold 
game came bark park 
tip dip big pig 
beat peat een den 


NON-RHYMING BUT SIMILAR 


sap sat peas peace 

iecaice raze save safe 

lake eee kit kid 

mad mat 

OPERATIONAL 

list course attack refuel 
time plot bingo cancel 
speed air report proceed 
dive fire distance label 
drop launch copy station 


A vocabulary listing in the order in which the words were 
trained is attached to the written instructions initially 
given to subjects and shown in Appendix C. 
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APPENDGx) B 


SUBJECT DATA SHEET 








Subject number: Name: Age: 
Time/date: Service: 
Rank: MOS (in words): 


Do you object to being taperecorded during the experiment? If 
you do, stop filling out this form and advise the experimenter 
now; otherwise, continue. 


How many hours experience have you had on voice recognition 
equipment in the last six months? 


hours (approximately) 


How many hours experience have you had on pursuit tracking 
devices in the past year? 


hours (approximately) 


Do you have a speech or hearing impediment? Yes No 
(circle one) 


Do you want a post participation briefing on your performance 
and on the hypotheses being tested by the experimenter? Note 
that if you request such a briefing, you must agree not to 
discuss this with anyone other than the experimenter so that 
no subject will learn what results are expected prior to his 
participation in the experiment; such prior knowledge could 
invalidate the results of the experiment. 


Yes No 
(circle one) 


After you have completed participation in the experiment you 
will be asked to write below any comments which you think may 
be useful to the experimenter. If you have any questions now, 
please ask the experimenter. Otherwise, give him this form 
now and start reading the pages titled "INTRODUCTORY REMARKS/ 
RECOGNIZER VOCABULARY TRAINING". 


POST EXPERIMENT COMMENTS 


(continue on reverse side if this space is insufficient) 


THANK YOU FOR YOUR PARTICIPATION 
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APPENDIX C 
WRITTEN INSTRUCTIONS 
INTRODUCTORY REMARKS / RECOGNIZER VOCABULARY TRAINING 

INTRODUCTORY REMARKS 

This experiment involves analysis of a combined human 
operator / voice recognition equipment system under various 
conditions of operator motor loading. The actual experiment 
will be carried out in a sound-proof booth and subject - 
experimenter communication during the actual experiment will 
be via the booth intercom system; however, you may remove the 
headset assembly during break periods and leave the booth. 

CAUTION: The mounting of the voice recognizer micro- 

phone on the headset assembly is very delicate, easily 

damaged, and difficult to repair. Please be careful 

while handling this assembly. 

Please carry out the experiment exactly as directed and 
do not discuss your performance with anyone other than the 
experimenter as inappropriate subject prior knowledge could 


invalidate the results. 


VOICE RECOGNIZER VOCABULARY TRAINING 

The 50 word vocabulary being used with the voice recog- 
nizer in this experiment is attached to these instructions. 
You will be required to repeat each word of this vocabulary 
ten times to train the recognizer to recognize your particular 
vocalizations of each word. To facilitate recognition by 


the voice recognizer, you should include in the ten repetitions 
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as many as possible of the different ways you might say the 
word in normal speech; for example, use different intonations 
and emphasis, and small variations in volume. 

In order to keep track of the number of times you say each 
word, and to reduce breath noise, it is best to speak the 10 
repetitions in several groups. For example, if the word is 
zero, it is better to group them as: 


000-000-0000 


Or 000-000-000-0 
rather than as 0000000000 
or 0-0-0-0-0-0-0-0-0-0 


Please observe the following guidelines while inputting 
voice data to the recognizer both during training and later 
during the actual experiment. 

a. Speak each word crisply and quickly but do not over- 
pronounce; for example, words ending in "t" - delete 
final "t" if more natural. 

b. Be sure to leave a distinct pause (specifically, at 
least one-tenth of a second of silence) between each 
word so that the recognizer can distinguish the end 
of one word from the beginning of the next. Sim- 
larly, do not leave a period of silence within a word 
or the recognizer will mistake it for two separate 
words. 

c. Avoid breathing into the microphone at the end of 
words as this will generate false inputs to the 


recognizer. 
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ad. Microphone location is very important and should be 
kept constant throughout the experiment; l1.e., adjust 
it if it gets out of place. The experimenter will 
initially demonstrate correct microphone placement. 
From this point on instructions will be given to you 


verbally by the experimenter. Please advise him if you have 


any questions now. 
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VOCABULARY LISTING 


attack 
list 
gale 
gale 
bingo 
sap 
sat 
time 
gold 
cold 
Caneel 
peas 
peace 
speed 
game 
came 
distance 
ace! 
raze 
copy 
bark 
park 
launch 
save 


safe 


(IN TRAINING ORDER) 
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49. 


refuel 
tip 
dip 
drop 
Fake 
Late 
course 
big 
pig 

Ee Oss 
kit 
kid 
plot 
beat 
peat 
proceed 
mad 
mat 
Fire 
Een 
den 
label 
air 
station 


dive 








A. 


APPENDIX D 
EXPERIMENTER AIDE-MEMOIRE 
I. PRELIMINARY CHECKS 
EQUIPMENT POWER 


Turn on master power Switch and ensure that the following 


devices are turned on and ready for uSe. 


B. 


the 


-voice processor 

-voice recognition system tape recorder 
=r 

-Miniterm 

-Maico audiometer 

-reel-to-reel tape recorder 

-cassette tape recorder 

-tracker and timing clocks (2) 

-booth lighting 


MINITERM 


Check that the Miniterm is in the "local" mode and that 
paper remaining is adequate for the next subject; then 


write this subject's name on top of the paper. 


ce 


and 


T600 


Read in E's experimental vocabulary tape then check T600 
Miniterm operation with several vocabulary words. First 


ensure that the microphone level control is set to "five". 


Ds. 


TRACKER AND TIMING CLOCKS 


Ensure that the circular track is in place and that the 


following controls are set as indicated. 


Tracker - Speed to 40 rpm 

- Motor to "reverse" (clockwise rotation) 

- Timer switch ON 

- Power switch ON 

- Lamp switch ON 

- Sensitivity control to "one o'clock" position 
Control clock - Buzzer switch ON 

- Outlet plug switch OPEN 


MAICO AUDIOMETER 
Ensure that the following controls are set as indicated. 


-Mixer control to BOTH-BOTH 
-monitor gain (L to 0 and pulled out, R to 6 and pushed in) 


71 








-talk-back gain to 5 

-talk-over gain to 3 

-output control to AIR (both L and R channels) 

-test stimulus control to TAPE (both L and R channels) 

-R volume control to 60, R meter ON, and R meter gain to 6 
-~L volume control to 0, L meter OFF 

-test presentation control to ON (both L and R channels) 


F. REEL-TO-~REEL TAPE RECORDER 


Ensure that the following controls are set as indicated. 
-channel to 1-4 

-monitor to TAPE 

oOo OLE 

~equalization to 7 1/2 

-tape speed to 7 1/2 

-~tape type to NORMAL (WIDE RANGE) 


Put on the tape used to signal S vocabulary words to 
repeat and check that the signal reaches both channels of the 
subject headset. Then reset the tape to the beginning of the 
practice section. 


G. CASSETTE TAPE RECORDER 


Ensure that the recorder is in the mono mode. Place a 
cassette into the recorder and record the subject number onto 
the beginning of the cassette using the booth microphone. 
Remember to turn on the cassette microphone mixer first and 
turn it off after. Check that the subject number was recorded 
properly. 


If. COMPLETION OF SUBJECT DATA SHEET AND INTRODUCTORY 
INSTRUCTIONS 


Write the subject number on the "Subject Data Sheet". 
Have the subject complete this sheet then answer any ques- 
tions he may have. 


Have the subject read the introductory instructions 
("Introductory Remarks / Recognizer Vocabulary Training"). 
While he is doing this check his data sheet. Answer any 
questions he may have. 
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TII. DEMONSTRATION OF T600 OPERATION AND LIMITATIONS 


Remove the T600 microphone (and the headset on which it 
1s mounted) from the booth and reconnect the microphone out- 
side the booth. Demonstrate to S the importance of the guide- 
lines that he just read, especially what happens if a pause is 
not left between words and what happens if a pause is left 
within a word. Let S "play with" the T600 in the recognition 
mode for several minutes. Be sure to tell him that the T600 
will recognize him much better after it has been trained with 
his versions of the vocabulary words. (E's versions are used 
during this demonstration.) Reconnect the T600 microphone and 
headset inside the booth after this demonstration. 
Note: be sure to demonstrate correct microphone placement to 
S before letting him "play with" the T600. 


IV. T600 VOCABULARY TRAINING 


Show S to the booth, seat him and explain the purpose of 
the two microphones as follows. 


"The microphone mounted on the headset is connected only 
to the T600. This other microphone is connected to a 
tape recorder so that I can record what you say during 
the experiment for analysis later; this microphone is 
live at all times and also permits me to hear what you 
say to me." 


Explain to S the procedure which will be used in vocabulary 
training as follows. 


"Next we will train the T600 with the 50 word vocabulary 
which you have attached to the ‘Introductory Remarks / 
Recognizer Vocabulary Training' sheets. Refer to this 
listing as we train the T600. The 50 words will be 
trained one at a time. I will prompt you with something 
like 'Enter the first word now, please’. Then you will 
enter your ten versions of the first word, keeping in mind 
the instructions which you just finished reading. As we 
get going I'll shorten the prompting to something like 
"Next word please’ to save time. 


"If you have trouble keeping track of the number of repeti- 
tions just keep repeating the word and I'll say 'stop' 
after the tenth repetition. Proper grouping as detailed 
in the instructions will help you keep count. 
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"Notice that some of the vocabulary words are similar. 

They have been paired and the differences underlined in 
the vocabulary listing for emphasis. Be sure to pro- 
nounce each of these words distinctly and correctly so 
that the T600 will be able to distinguish them. To obtain 
adequate recognition performance from the T600 it may be 
necessary for you to pronounce some words a little more 
distinctly than you normally do, both in training and 
later for recognition. However, the key word here is 
‘little'; speak as naturally as possible. 


"I'll be turning the microphone switch on and off for each 
different word, so if you want to clear your throat or 
Say something to me do so a couple of seconds after com- 
pleting the tenth version of a word; otherwise, keep 
Sllent between the training of different vocabulary words. 


"If any word needs retraining I'll advise you why and we 
will retrain the word immediately, rather than waiting 
until the end of the vocabulary. 


"Here's a special pen which will write on the plastic 
covering your copy of the vocabulary. Use it to check off 
words as we finish training them. Please return it to me 
after the vocabulary has been trained and tested. 

"We can take a break after word number 24 if you wish; I'll 
check with you at that time. Training the entire vocab- 
ulary will take about 20 minutes. Do you have any 
questions?" 


Remind S of the delicate mounting of the T600 microphone 
on the headset. Have him put on the headset and ensure that 
he positions the microphone correctly. Advise him that you 
will contact him over the headset then leave the booth and 
close the door. 


Put on your headset and contact S using the talk-over 
switch and microphone of the Maico audiometer. Adjust the 
talk-over gain as necessary so that S can clearly hear what 
you say. 


Train each word of the vocabulary as follows. 

-enter "CNTRL-U", the word number, and a carriage return 
on the T600 control 

-confirm that the correct word is called for 

-turn on the T600 microphone switch 

-press the talk-over switch and prompt S to train the 
word ("next word, please") 
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-after he has entered his tenth version of the word turn 
off the T600 microphone switch (and tell him to stop if 
he is still repeating the word) 

-1f the word has to be retrained, tell S why and retrain 
it; else, 

-~check off the word on your copy of the vocabulary and 
train next word. 


Ask S if he wishes to take a rest break after word number 
24 has been trained. Take a break if he does; otherwise, 
finish training the vocabulary and then take a rest break. 


V. RECOGNITION CHECK / RETRAINING (AS NECESSARY) 
AND WORD REPETITION PRACTICE 


Explain to S that next the vocabulary will be checked for 
recognition and that words will be retrained as necessary and 
that after that he will be given a practice session involving 
repeating a sequence of vocabulary words, one word at a time, 
that will be presented to him via his headset. 


"Next we'll check that the T600 now recognizes your ver- 
Sions of the 50 vocabulary words. We'll check the words 
one at a time. Any word which is not correctly recog- 
nized at least twice in three tries will be retrained. 
I'll be prompting you as follows. If I say 'next', say 
the next word three times. If the T600 correctly recog- 
nized at least two of the three repetitions, I'll say 
"next" and we'll go on to the next word. If it did not, 
I'll ask you to retrain the word before we continue. 
Then we'll recheck recognition and continue or retrain 
again aS appropriate. If either word of a rhyme or non- 
rhyming but similar word pair requires retraining, we'll 
recheck recognition of both words of the pair and retrain 
and recheck as necessary before continuing. 


"When we've finished checking and retraining the vocabulary, 


I'll play a 2 1/2 minute recording of a random ordering 
of the vocabulary words to you. Words are three seconds 
apart and you are to repeat each word, one at a time, for 
recognition by the T6000. Try to repeat each word. If 
you are uncertain of a word, guess with a word in the 
vocabulary. Before you start hearing words to repeat, 
you will hear the warning 'this part will begin in five 
seconds' and then five seconds later you will hear the 
first word that you are to repeat. At the end you will 
hear ‘end of this part’. At that time we will take a rest 
break. Do you have any questions?" 
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Show S to the booth and have him put on the headset. 
Check that he has the T600 microphone positioned correctly then 
leave the booth and close the door. Put the T600 in recogni- 
tion mode ("CNTRL-W", "N", then carriage return), put on your 
headset, and confirm that S is ready. 


Turn on the T600 microphone switch and ask S to say the 
first word. Then check recognition of the vocabulary words, 
retraining where necessary. 


When all vocabulary words have been checked for recognition 
and retrained as necessary, ensure that S is ready to start 
repeating the words recorded on the practice tape. If he is, 

-put the T600 in recognition mode ("CNTRL-W","N", and 

carriage return) 

-turn on the cassette tape recorder microphone mixer (two 

LH-most controls to 12 o'clock) 

-start recording on the cassette tape recorder 

-start the practice tape on the reel-to-reel tape recorder 

-when you hear the 5 second warning start, turn on the 

T600 microphone switch. 


When the end of the practice tape is reached: 

-advise S to remove the headset, leave the booth, and take 
a break 

-stop the cassette tape recorder and ensure that it re- 
corded properly 

-turn off the cassette tape recorder microphone mixer 
(two LH-most controls fully counter-clockwise) 

-T600 microphone switch to OFF 

-rewind the practice tape back to the beginning of the 
practice session. 

-ask S if the volume was adequate and adjust it as 
necessary 

-answer qny questions that S may have. 


VI. EXPLANATION OF EXPERIMENTAL CONDITIONS 
Explain the experimental conditions to S as follows. 


During the actual experiment, you will be exposed to three 
different conditions of motor loading, which I'll explain 
later. Each condition will last five minutes and there 
will be a short break after each condition. 


"During all three conditions you will be required to repeat 
vocabulary words presented to you exactly as during the 
practice you just finished. The sequences of words will 
be different but the pacing is the same, i.e., a word 
every three seconds, and the same warnings are used at the 
beginning and end. Remember, you are repeating these 
words for recognition by the T600. 
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"During two of the conditions you will be simultaneously 
tasked on the pursuit tracker. A different tracker task 
will be used during each of these two conditions and these 
will be explained in a couple of minutes. During the 
third condition all you are to do is repeat the words you 
hear for recognition by the T600. 


"In all three conditions repeating the words for recognition 
by the T600 is the more important task and you are to re- 
peat each word, guessing with a word from the vocabulary 
if you are uncertain, just as you did during the practice 
you just completed." 


VII. TRACKER BRIEFING AND PRACTICE 


Tell S the names of the two tracker tasks and the three 
experimental conditions as follows. 


"The two tracker tasks which you will be performing are 
called Circular Tracking Task (CTT) and Square Tracking 
Task (STT); the names of the corresponding experimental 
conditions are the same. The third experimental condition, 
during which you only have to repeat the words you hear, 
is called No Tracking Task (NTT). 


Explain the tracking tasks to S as follows. 


"During the tracking tasks you will be required to try to 
keep the head of the tracker wand (show S the wand and 
the wand head) positioned over the light which will move 
around either a circular or square path, depending on the 
task (show S the circular path and the light). In both 
cases the angular velocity is a constant 40 rpm. In the 
Circular Tracking Task, because the path is a circle, 
the light will travel at a constant speed. However, in 
the Square Tracking Task, because the path is a square, 
the light will not travel at constant speed; it will 
travel faster nearer the corners than on the centers of 
the sides. 


"In both cases a stop clock will automatically record the 
total time during which you keep the wand head positioned 
over the light. This is called time on target (TOT) and 
will be your tracker score. You are to attempt to maxi- 
mize this. 


"Both tracking tasks are to be performed while standing; 


you are to sit for the No Tracker Task condition. You 
may hold the wand anyway you wish. 
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"It's easy to cheat at these tracking tasks. For example, 
you can change the angular velocity (show S how) or simply 
hold the wand head up to the overhead light to "fool" the 
Stop clock. Please follow the instructions that I give 
you and don't cheat." 


"I do not require you to achieve any particular proficiency 
level on either of the Tracker tasks. It is sufficient 
that you understand the tasks and perform them to the best 
of your current abilities. However, I do require that you 
do this for the full five minutes of each condition; i1.e., 
don't get frustrated, say with the Square Tracking Task, 
and completely stop tracking and only repeat the words 
that you hear. You must keep trying to maximize your TOT 
scores. 


Advise S that he now has a couple of minutes to practice 
on the tracker using the circular path. Zero out the TOT 
peitock. ~oet the control clock to 2 1/2 minutes and start it 
to start the tracker for the practice. After the practice 
tell S his TOT score. 


VIII. COMBINED TRACKER / WORD REPETITION PRACTICE 


When S has finished his practice on the tracker explain 
to him that he will next be given a 2 1/2 minute combined 
Tracker / Word Repetition practice, as follows. 


"You will next be given a combined Tracker / Word Repetition 
practice. I will play the same 2 1/2 minute recording of 

a random ordering of the vocabulary words which you heard 
in the first practice. You are to repeat these words, one 
at a time as before, for recognition by the T600. As be- 
fore, the tape begins with the warning 'this part will 
begin in five seconds! and five seconds later you will 

hear the first word that you are to repeat; at the end 

of the tape you will hear ‘end of this part’. 


"While you are repeating the words you hear, you are to 
Simultaneously perform the Circular Tracking Task as well 
aS you can, i.e., try to maximize your TOT score. Remem- 
ber though that repeating the words for recognition by 
the T600 is the more important task and that you are to 
try to repeat each word that you hear, guessing with a 
word from the vocabulary if you are uncertain, just as 
before. 


"Tf you want me to adjust the volume of the headset signal, 
i.e., the words for repetition, be sure to let me know 
right after this practice; the volume must be kept con- 
stant during the three conditions of the actual experiment. 
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"We will take a rest break after this practice. Do you 
have any questions?" 


Show S to the booth and have him put on the headset. 
Check that he has the T600 microphone positioned correctly 
then leave the booth and close the door. 


Ensure that the TOT clock is zeroed. Put on your headset 
and confirm that S is ready. If he is, 
-put the T600 in recognition mode ("CNTRL-W", "N", and 
carriage return) 
-set the control clock to 2 1/2 minutes 
-turn on the cassette tape recorder microphone mixer 
(two LH-most controls to 12 o'clock) 
-start recording on the cassette tape recorder 
-start the practice tape on the reel-to-reel tape recorder 
-turn on the T600 microphone switch 
-when you hear the first word for repetition start the 
tracker by turning the control clock ON 


Monitor S's word repetition and TOT score to ensure that 
S is adhering to instructions. Ensure that the T600 is func- 
tioning properly. 


When the end of the practice tape is reached: 

-turn the control clock OFF 

-advise S to remove the headset, leave the booth and 
take a break 

-stop the cassette tape recorder and ensure that it 
recorded properly 

-turn off the cassette tape recorder microphone mixer 
(two LH-most controls fully counter-clockwise) 

-T600 microphone switch to OFF 

-position the reel-to-reel tape at the beginning of the 
first experimental condition section 

-ask S if the volume was adequate and adjust it as 
necessary 

-tell S his TOT score and explain to him that he will not 
receive any more feedback on his performance until the 
end of the experiment 

-zero out the TOT clock 

-answer any questions that S may have 

-advance the Miniterm paper several lines 


While S is taking his break fill in the top line of his 
T600/Subject Performance Summary Sheet (sample attached to 
this aide-memoire) and the top line of his three Subjective 
Fatigue Checklists. Turn them over so that S can not deter- 
Mine in advance the order of condition presentation. 
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IX. ACTUAL EXPERIMENT 


For each experimental condition, install the appropriate 
tracker "path". Advise S what condition is next (and only 
what condition is next, i.e., not the whole sequence of 
conditions). Advise him that the condition will last for 
five minutes and remind him that repeating the words for 
recognition by the T600 is the more important task, to try to 
respond to each word, and to guess with a word from the vocab- 
ulary if he is uncertain. 


Show S to the booth and have him put on the headset. 
Check that he has the T600 microphone positioned correctly 
then leave the booth and close the door. 


Ensure that the TOT clock is zeroed. Put on your headset 
and confirm that S is ready. If he is, 

-put the T600 in recognition mode ("CNTRL-W", "N", and 
carriage return) 

-~set the control clock to 5 minutes 

-turn on the cassette tape recorder microphone mixer (two 
Piemost COntrols to 22° 6 clock) 

-start recording on the cassette tape recorder 

-start the tape on the reel-to-reel tape recorder 

-turn on the T600 microphone switch 

-when you hear the first word for repetition start the 
tracker by turning the control clock ON (as applicable) 


Monitor S's word repetition and TOT score to ensure that 
S is adhering to instructions. Ensure that the T600 is func- 
tioning properly. Write the name of the experimental condi- 
Gioneiec., NIT, CTT, or STT) on the Miniterm output. 


When the end of the tape recording for the experimental 

condition is reached: 

eum thie control clock OFF 

-turn off the T600 microphone switch 

~advise S to wait in the booth and that he may remove 
the headset 

-stop the cassette and reel-to-reel tape recorders 

-go to the booth with a copy of the Subjective Fatigue 
Checklist Instructions and S's Checklist for this con- 
dition and tell S "Please f1i1l1 out this checklist in 
accordance with these instructions." 

-copy the TOT onto S's T600/Subject Performance Summary 
Sheet while S fills out the checklist then zero out the 
TOT clock and turn over the Performance Summary sheet 
so that S can not determine his TOT score 

-ensure that the cassette tape recorder recorded properly 
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-turn off the cassette tape recorder microphone mixer 
(two LH-most controls fully counter-clockwise) 

-position the reel-to-reel tape at the beginning of the 
next experimental condition section 

-advance the Miniterm paper several lines. 


When S has filled out the Subjective Fatigue Checklist, 
take it from him and advise him that he may take a short 
break before he starts the next condition. Do not tell him 
what the next condition will be at this time. 


If the condition completed was the last condition, give 
S back his Subject Data Sheet so that he can record any com- 
ments he may have. When he returns this, tell him his TOT 
scores and brief him on the hypotheses being tested in the 
experiment (depending on his wishes as indicated on his 
Subject Data Sheet), answer any questions he may have and 
thank him for participating. 
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SUBJECT NUMBER 


MN & WN FE 


ORDER OF PRESENTATION 


Cir 
STT 
Nie 
Cle 
STT 
Nae 


Cir 
NTT 
NTT 
Sreds 
CrT 
oat 


NTT 
eel 
NTT 
eri 
STT 
Crt 


NTT 
Cre 
Sat 
oLt 
Cry 
NTT 


APPENDIX E 
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ORDER OF PRESENTATION OF CONDITIONS 


NET 
Ne 
CT 
Sr 
Cir 
STT 


NTT 
Save 
Cre 
eae 
ont 
NTT 


ont 
CTT 
CLT 
Sor 
NTT 
Neer 


rb 
NTT 
NTT 
Cr 
Sat 
Cre 


OF CONDITIONS 


Sells 
CRY 
ott 
Nee 
NTT 
Cre 


ott 
CTs 
led 
NTT 
NTT 
Cre 


Cla 
NTT 
Sn 
NTT 
Cre 
Seng 


Cra 
Sia 
CAVE 
NTT 
NTT 
Sele 





Poe LCE 


maze 
pig 

mat 
aneit 
course 
COpy 
station 
dive 
peat 
wlot 


fat 
launch 
attack 
park 
save 
ten 
eon 
came 
sat 
gold 


report 
game 
safe 
dip 
label 
den 
cold 
time 
fire 
ete e 


tip 
beat 
cancel 
lake 
mad 
kid 
peas 
proceed 
cave 
big 


APPENDIESE 


RANDOM ORDERINGS OF THE 


tino lL CONDI T EON 


ten 
dip 
label 
den 
game 
air 
gold 
speed 
attack 
beat 


ace} 
proceed 
tip 
cancel 
sat 
fire 
launch 
ame 
Fate 
course 


peat 
Cake 
station 
mat 
big 

pig 

sap 
came 
raze 
save 


COpy 
list 
Dingo 
drop 
refuel 
peas 
sate 
lake 
mad 
cold 


list 
ele 
park 
distance 
kid 
report 
Lay ees 
save 
gale 
drop 


bingo 
raze 
came 
sap 

pig 
race 
cancel 
big 
station 
air 


peas 
ten 
game 
lake 
mat 
attack 
dip 
safe 
Pec 
den 


peat 
peace 
course 
bark 
label 
dive 
fire 
sat 
tip 
kat 
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VOCABULARY 


SECOND CONDITION 


gale 
fab 
big 
race 
ake 
time 
course 
game 
ten 
mat 


list 
eae 
cold 
Pate 
mad 
proceed 
label 
dip 
safe 
attack 


speed 
dive 
beat 
bark 
refuel 
park 
kid 
icate 
raze 
kit 


Sat 
EEOOrt 
station 
sap 
save 
distance 
cancel 
launch 
air 
Peace 


ten 
tale 
den 
cold 
list 
time 
sap 
launch 
gale 
deop 


station 
distance 
COpy 
peas 
pig 

Hate 
peace 
dip 

tip 
report 


gold 
beat 
sat 
bingo 
kid 
cancel 
game 
raze 
label 
bark 


ee 
park 
speed 
kit 
Pear 
save 
aerack 
came 
lake 
big 








Peace 
gate 
bark 
late 
drop 
speed 
distance 
refuel 
sap 
bingo 


pig 
park 
attack 
peat 
peace 
sat 
beat 
list 
Face 
speed 


cen 
time 
gold 
peas 
report 
cold 
den 
cancel 
course 


COpy 


distance 
station 
label 
raze 
drop 
Siaeee 

sap 
game 
ieare 
lake 


dive 
time 
peace 
plot 
Kate 
report 
park 
distance 
bark 
kid 


THIRD CONDITION 


came 
beat 
speed 
pig 
cancel 
sap 
distance 
cOpy 
station 
course 


dip 
label 
raze 
list 
park 
Fate) 
game 
save 
lake 
proceed 


ake 
eels) 
oul 
launch 
peas 
time 
gale 
report 
gold 
cen 


speed 
refuel 


beat 


proceed 


Gold 


gold 
time 
mad 


COpy 


launch 
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den 
peat 
bingo 
came 
peas 
pilot 
COpy 
pig 
drop 
gold 


dip 
came 
launch 
bingo 
mad 
big 
save 
air 
plot 
refuel 


mat 
fire 
dive 
sale 
kit 
tip 
mace 
bark 
proceed 
kid 


mad 
dive 
course 
fire 
mat 
refuel 
plot 
air 
proceed 
Scene 


mad 
air 
plot 
dive 
peace 
Sale 
kid 
drop 
refuel 
fire 


den 
bark 
bingo 
mat 
big 
Sate) 
kit 
ED 
peat 
attack 





APPENDIX G 
SUBJECTIVE FATIGUE CHECKLIST 
Subject number Experimental condition 


FEELING TONE CHECK LIST 


Better Same Worse 








No. than as than SEaeenenc 
1. (on) ( ) (  ) slightly tired 
ons (  ) (ee) Co) like I'm bursting with energy 
er, ( ) (  ) (a) extremely tired 
4. (  ) (  ) ( ) quite fresh 
oy ( ) ( ) ( ) slightly pooped 
6. iP ) (  ) (  ) extremely peppy 
ue ( ) ( ) ( ) somewhat fresh 
Se (  ) (  ) or) petered out 
9. Go) ( ) (  ) very refreshed 
LOR (one) Ce) (ia) ready to drop 
el. ,.) (  ) (ay fairly well pooped 
ne. = * (ram) Cc) very lively 
ese ( ) ( ) ( ) very tired 


Have you checked each statement? 
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INSTRUCTIONS FOR COMPLETING FEELING TONE CHECKLIST 


People feel different at various times for various reasons. 
Some arise after a night's rest feeling "quite rested" while 
others may feel "a little tired". A hard day's work ora 
Vigorous workout at the gym may make you feel "fairly well 
pooped"; yet, a shower, a cup of coffee, or merely a few 
minutes relaxing in a comfortable chair may make you feel 
"very refreshed". 


I would like to find out how you feel right now. On the 
accompanying sheet, you will see 13 statements which describe 
different degrees of freshness or peppiness and tiredness. For 
each statement you will have to determine in your own mind 
Piaierier you feel at this instant (1) "Better than", (2) the 
"Same as", or (3) "Worse than" the feeling described by that 
statement. Having done this you will then place an "X" in the 
appropriate box. 


Consider the following example: 


Better Same Worse 
No. than as than Statement 


0. (  ) (  ) (  ) somewhat tired 


If right now you felt “somewhat tired" you would place an 
"X" in the box marked "Same as". If, however, you felt fresh 
or full of pep you would check the box marked "Better than" 
because you would be feeling better than "somewhat tired”. 
On the other hand, if you felt exhausted you would place an 
"X" in the box marked "Worse than". 


Take each statement in order; do not skip around from one 
to another. Read each statement carefully so that you under- 
stand what it means. It may help you to understand some state- 
ments if you mentally insert the words "I feel" or "I am" 
before the statement. 


This is not a test. You have all the time you need. 
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APPENDIX H 


this is because of inaccuracy in the elements of the row caused by rounding to two 


indicates the proportion of the time that the T600 "thought" that a subject said word j when the subject 
The last column, annotated “beep”, indicates the proportion of the time that the T600 rejected vocabulary 


ij 


actually said word 1. 
Several rows do not sum to exactly 1.00; 


decimal places. 


word i and “beeped". 


The matrix element a 


CONFUSION MATRIX FOR OPERATOR MOTOR LOADING - EXPERIMENTAL CONDITION NTT 
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APPENDIX I 


CONFUSION MATRIX FOR OPERATOR MOTOR LOADING -—- EXPERIMENTAL CONDITION CTT 


. indicates the proportion of the time that the T600 “thought" that a subject said word 3 when the subject 


1j 


The matrix element a 
actually said word i. 


indicates the proportion of the time that the T600 rejected vocabulary 


The laste column, annotated “beep", 


word i and “beeped”. 


Several rows dO not sum to exactly 1.00; this 1s because of inaccuracy in the elements of the row caused by rounding to two 


decimal places. 
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APPENDIX J 


CONFUSION MATRIX FOR OPERATOR MOTOR LOADING - EXPERIMENTAL CONDITION STT 


the time that the T600 "thought" that a subject said word j when the subject 


. indicates the proportion of 


aj 


The matrix element a4 
actually said word i. 


"beep", indicates the proportion of the time that the T600 rejected vocabulary 


The last column, annotated 


word 1 and "beeped”. 


Several rows do not sum to exactly 1.00; this is because of inaccuracy in the elements of the row caused by rounding to two 


decimal places. 
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APPENDIX K 


CONFUSION MATRIX FOR ALL OPERATOR MOTOR LOADING - EXPERIMENTAL CONDITIONS COMBINED 


indicates the proportion of the time that the T600 “thought" that a subject said word j when the subject 
The last column, annotated "beep", indicates the proportion of the time that the T600 rejected vocabulary 


The matrix element a.. 


S| 


actually said word 1 


word i and “beeped”. 


Several rows do not sum to exactly 1.00; this 1s because of inaccuracy in the elements of the row caused by rounding to two 


decimal places. 
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APPENDIX L 
T600 RECOGNITION ERRORS* 


An entry w/x (y/z) indicates that w recognition errors, of 
which y were errors of rejection (of vocabulary words), occur- 
red in the first half of the trial and x recognition errors, 
of which z were errors of rejection (of vocabulary words), 
occurred during the second half of the trial. 


I. TOTAL T600 RECOGNITION ERRORS (I.E., INCLUDING ALL 
VOCABULARY WORD TYPES) 


OPERATOR MOTOR LOADING - EXPERIMENTAL CONDITION 


SUBJECT 
NUMBER _ NTT CTT sTttT 
i: 97/150 (0/2) 9/6 (3/0) 9/8 (1/7) 
2 6/4 (7A) eya0 i 272) 4/9 (178) 
3 5/9 (i573) 8/11 (0/1) 17/10 (278 
4 3/4 (0/1) 375 (0/0) 4/4 (0/0) 
5 8/6 (0/0) TOye2 “ (071) 10/12 (0/1) 
6 By 2 (1/0) 2/4 (0/0) by 3 (0/0) 
q 8/8 (ay) 10779 (1/0) 2s (0/0) 
8 4/5 (0/0) 6/7 (1/1) 6/9 (0/0) 
9 a2 (0/0) 6/6 (0/1) oy, (0/0) 
10 2/0 (0/0) 9712 (371) 4/7 (Ga) 
i. 7/8 (iyi) 8/12 (0/1) LO/12 (O72 
12 8/5 (2715) 12/14 (1/0) 12/11 (47) 
13 0/2 (0/0) 4/7 (1/2) 3/4 (0/1) 
14 5/5 (0/1) 5/9 (0/0) 6/5 (077) 
15 3/8 (07 il) 277 (0/1) 9/8 (6/4) 
16 7/8 (0/0) 10/5 (0/0) 8/10 (0/0) 
Ly 6/5 (0/0) 8/5 (170) 6/9 (aie) 
18 3/5 (0/0) 3/3 (1/1) 6/0 (1/0) 
19 8/6 (0/0) 6/4 (0/0) 7/8 (0/0) 
20 4/6 (1/0) L2kO. P22) 6/7 (0/0) 
D1. 10/8 (0/0) 8/9 (0/1) 14/10 (0/1) 
22 3/3 (0/0) 4/5 (1/0) 2/6 (0/0) 
23 6/6 (0/0) 5/7 (07) 373 (0/0) 
24 3/4 (OVAln) 3/6 (0/0) 8/5 (1/0) 
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II. T600 RECOGNITION ERRORS ON RHYMING VOCABULARY WORDS 


OPERATOR MOTOR LOADING - EXPERIMENTAL CONDITION 


SUBJECT 
NUMBER NIT Gar STT 
d. 4/8 (O/1) 6/4 (27.0) 5/6 (1) 
2 4/2 (0/0) 7/8 (v7.5) 3/6 (aya) 
3 3/5 (e772) 3/8 (0/1) 7/7 (i772) 
4 73 (O71) 2/4 (0/0) 273 (0/0) 
2 4/4 (0/0) 5/6 (0/0) 4/6 (0/0) 
6 WyAal (0/0) Ibs) (0/0) 4/3 (0/0) 
7 5/6 (0/0) 4/3 (0/0) 8/4 (0/0) 
8 3/4 (0/0) 373 a) a7 (0/0) 
9 1/2 (0/0) 575 (0/1) 5/3 (0/0) 
10 2/0 (0/0) 370 (0/0) 87 3 (1970) 
EL 6/5 (1/0) 4/7 (0/0) a9 (0/1) 
2 4/4 (271) 7/6 (1/0) 4/7 ( 27722 ) 
3 0/2 (0/0) 4/5 (ral) 2/2 (0/1) 
14 5/3 (0/0) 4/8 (0/0) 5/4 (0/0) 
15 2/5 (0/0) 1/4 (0/0) 4/5 (2/2) 
16 5/4 (0/0) 6/2 (0/0) 5/5 (0/0) 
-/ 4/4 (0/0) 5/4 (0/0) 5/5 (1/2) 
18 1/2 (0/0) 2/2. (1/0) 4/0 (0/0) 
19 6/4 (0/0) 4/3 (0/0) 3/3 (0/0) 
20 275 (1/0) 5/6 (0/0) B/D (0/0) 
21 8/7 (0/0) Gy > (0/1) 9/5 (O72) 
Zz 73 (0/0) 373 (1/0) 275 (0/0) 
23 3/4 (0/0) 4/4 (0/1) 275 (G70) 
24 273 (0/0) 3/4 (0/0) 6/2 (1/0) 
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T600 RECOGNITION ERRORS ON NON-RHYMING BUT SIMILAR 


OPERATOR MOTOR LOADING - EXPERIMENTAL CONDITION 
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IV. T600 RECOGNITION ERRORS ON OPERATIONAL VOCABULARY WORDS 


OPERATOR MOTOR LOADING - EXPERIMENTAL CONDITION 


SUBJECT 
NUMBER _ NTT CTT sTT 

ih 1/0 (0/0) 78 (0/0) 2/0 (0/0) 
Zs val (1/0) 2 (1/1) 0/1 (0/0) 
3 2 (0/1) 2/0 (0/0) ye (170) 
4 0/0 (0/0) 0/0 (0/0) 0/0 (0/0) 
> 0/0 (0/0) 0/1 Or 2) 2/1 (0/0) 
6 70 (1/0) 0/0 (0/0) 1/0 (0/0) 
7 0/1 (077A) PAVEN (1/0) O71: (0/0) 
8 0/0 (0/0) Al (0/0) 1/2 (0/0) 
9 0/0 (0/0) 0/1 (0/0) 1/0 (0/0) 
10 0/0 (0/0) 3/3 (2/1) Oval (0/0) 
el 1/2 (0/1) 2/3 (0/1) 2 al (0/1) 
fie 0/0 (0/0) 1/2 (0/0) 4/2 (2/2) 
Le 0/0 (0/0) 0/1 (Gyan) 1/0 (0/0) 
14 0/2 (0/1) 0/0 (0/0) 0/0 (0/0) 
15 oval (0/1) 0/1 (0/1) 2/0 (2770) 
16 0/0 (0/0) 0/0 (0/0) Oil (0/0) 
a7 1/0 (0/0) 0/0 (0/0) 0/1 (0/0) 
18 ly 2 (0/0) iy Cova) 2/0 (1/0) 
19 0/0 (0/0) 0/0 (0/0) 0/0 (0/0) 
20 0/1 (0/0) 3/2 (272) ea (0/0) 
Za Oya (0/0) 2 (0/0) 2773 (0/0) 
22 0/0 (0/0) 0/0 (0/0) 0/0 (0/0) 
23 1/0 (0/0) 0/0 (0/0) 0/0 (0/0) 
24 O72 (0/1) 0/0 (0/0) 0/1 (0/0) 


* A T600 recognition error was operationally defined in this 
research to be a failure of the T600 to recognize correctly 
any vocabulary word which S spoke and includes both in- 
correct recognition and rejection of vocabulary words; 

T600 recognition errors do not include those cases where S 
spoke a word not in the vocabulary (or coughed, sighed, 
etc.) and the T600 generated a recognition. 
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APPENDIX M 
SUBJECi WE RbAL ERRORS* 


An entry w/x (y/z) indicates that a total of w subject verbal 
errors, of which y were errors of not speaking any word or 
Speaking a non-vocabulary word (when prompted with a vocab- 
ulary word), occurred in the first half of the trial and a 
total of x subject verbal errors, of which z were errors of 
not speaking any word or speaking a non-vocabulary word (when 
prompted with a vocabulary word), occurred in the second half 
m2 the trial. 


OPERATOR MOTOR LOADING - EXPERIMENTAL CONDITION 


SUBJECT 
NUMBER _ NTT ou STT 
1 2] (27) 0/1 (0/1) 0/0 (0/0) 
2 0/0 (0/0) 70 (070) WZ (ae) 
3 0/70 (0/0) 0/0 (0/0) 0/0 (0/0) 
4 2/0 (270) 0/0 (0/0) 0/0 (0/0) 
2 0/1 (0/0) 0/0 (0/0) 272 (2/2) 
6 070 (0/0) 0/0 (0/0) 0/0 (0/0) 
7 0/0 (0/0) 0/0 (0/0) 0/0 (0/0) 
8 0/0 (0/0) 0/0 (0/0) 0/0 (0/0) 
9 0/0 (0/0) 0/0 (0/0) y/o (0/0) 
10 0/0 (0/0) O70 (0/0) 0/1 (0/1) 
Le 0/0 (0/0) 0/0 (0/0) 0/0 (0/0) 
2 0/0 (070) 07.0 (0/0) 0/0 (0/0) 
i 0/0 (0/0) 0/0 (0/0) 0/0 (0v0) 
14 0/0 (07 0) 0/0 (0/0) 0/0 (07 0} 
ILS 0/0 (07 OD 0/0 (0/0) 0/0 (0/0) 
16 0/0 (0/0) 0/0 (0/0) 0/0 (0/0) 
iy 0/0 (0/0) 0/0 (0/0) 0/0 (0/0) 
es 0/0 (0/0) 0/0 (0/0) 0/0 (0/0) 
ao 0/0 (0/0) 0/0 (0/0) 0/0 (070) 
20 0/0 (0/0) 0/0 COy.0)) 0/0 (0/0) 
21 0/0 (0/0) 1/0 (17 0) (wal (171) 
22 0/0 (0/0) 0/0 (0/0) 1/0 (1/0) 
23 0/0 (0/0) O7 LT COVE) 0/0 (0/0) 
24 O7 i (O77 am) 0/0 (0/0) 0/0 (0/0) 


* A subject verbal error was defined in this research to be a 
failure of a subject to repeat correctly the presented 
vocabulary word. This failure could be either a failure to 
respond (omission) or responding with a non-vocabulary word 
or the wrong vocabulary word (commission). 
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APPENDIX N 
TRACKER TIME ON TARGET SCORES (MINUTES:SECONDS) 


OPERATOR MOTOR LOADING - EXPERIMENTAL CONDITION 


SUBJECT 
NUMBER ey San 
i: 32h 79 ee SZ 
2 Ze D5 1:05.72 
3 1:36.84 Ie Oi0k7S 
4 3204.01 1:50.91 
5 2:45.81 es oe 
6 4:36.08 ies ee le2 
7 2.2 20 1:14.84 
8 2-509 Ie 3 772 
9 1:56.89 iene OZ 
10 2255.06 1:37.68 
1 isco 3129 
12 2205.59 O52 0 
13 5219.62 a eS 
14 2:39.04 ie Z20m 60 
15 2:29.46 126 23 O 
16 ao 6S eee ae 
17 era oa, eee Ses 
18 2200-56 1:16.44 
19 S095 55.33 
20 2:43.67 1:32.01 
21 221305 lel lesen 
22 PEIN BS, Nae sas 8. 
23 Ze ae ela 
24 1:59.16 ieOie2 | 


oi 





APPENDIX O 


SUBJECT EVE, -ALTGUE SCORES * 


OPERATOR MOTOR LOADING - EXPERIMENTAL CONDITION 


SUBJECT 
NUMBER _ NET eum Str 
1 14 nS i 
Z Z LA eZ 
3 15 13 3 
4 20 ey 16 
5 16 18 19 
6 13 eZ 3 
7 14 14 14 
8 ES 15 13 
9 12 EL ll 
10 14 14 14 
ital 13 13 9 
eZ 12 lez ee 
13 13 a2 12 
14 12 13 13 
> 16 12 14 
LG 5 9 3 
17 VS ae 13 
18 25 2 25 
19 14 eZ cS 
20 18 18 18 
21 13 13 ES 
22 14 14 14 
23 iS 16 2 
24 14 14 ll 


* Higher scores are associated with lower subjective fatigue 
and vice versa. 


Scores were obtained by multiplying the number of items scored 
as "better than" by two and adding the number of items scored 
as "same as", as recommended by those who developed the check- 
list (Pearson and Byars, 1956). 
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